Goto

Collaborating Authors

 event stream


EV-Eye: Rethinking High-frequency Eye Tracking through the Lenses of Event Cameras

Neural Information Processing Systems

In this paper, we present EV-Eye, a first-of-its-kind large-scale multimodal eye tracking dataset aimed at inspiring research on high-frequency eye/gaze tracking. EV -Eye utilizes the emerging bio-inspired event camera to capture independent pixel-level intensity changes induced by eye movements, achieving sub-microsecond latency.



Continuous Spatiotemporal Events Decoupling through Spike-based Bayesian Computation

Neural Information Processing Systems

Numerous studies have demonstrated that the cognitive processes of the human brain can be modeled using the Bayesian theorem for probabilistic inference of the external world. Spiking neural networks (SNNs), capable of performing Bayesian computation with greater physiological interpretability, offer a novel approach to distributed information processing in the cortex. However, applying these models to real-world scenarios to harness the advantages of brain-like computation remains a challenge. Recently, bio-inspired sensors with high dynamic range and ultra-high temporal resolution have been widely used in extreme vision scenarios. Event streams, generated by various types of motion, represent spatiotemporal data.


Exploiting Spatiotemporal Properties for Efficient Event-Driven Human Pose Estimation

Zhou, Haoxian, Xu, Chuanzhi, Chen, Langyi, Chen, Haodong, Chung, Yuk Ying, Qu, Qiang, Chen, Xaoming, Cai, Weidong

arXiv.org Artificial Intelligence

Human pose estimation focuses on predicting body keypoints to analyze human motion. Event cameras provide high temporal resolution and low latency, enabling robust estimation under challenging conditions. However, most existing methods convert event streams into dense event frames, which adds extra computation and sacrifices the high temporal resolution of the event signal. In this work, we aim to exploit the spatiotemporal properties of event streams based on point cloud-based framework, designed to enhance human pose estimation performance. We design Event Temporal Slicing Convolution module to capture short-term dependencies across event slices, and combine it with Event Slice Sequencing module for structured temporal modeling. We also apply edge enhancement in point cloud-based event representation to enhance spatial edge information under sparse event conditions to further improve performance. Experiments on the DHP19 dataset show our proposed method consistently improves performance across three representative point cloud backbones: PointNet, DGCNN, and Point Transformer.


A Modular Framework for Rapidly Building Intrusion Predictors

Wang, Xiaoxuan, Stadler, Rolf

arXiv.org Artificial Intelligence

Abstract-- We study automated intrusion prediction in an IT system using statistical learning methods. The focus is on developing online attack predictors that detect attacks in real time and identify the current stage of the attack. While such predictors have been proposed in the recent literature, these works typically rely on constructing a monolithic predictor tailored to a specific attack type and scenario. Given that hundreds of attack types are cataloged in the MITRE framework, training a separate monolithic predictor for each of them is infeasible. In this paper, we propose a modular framework for rapidly assembling online attack predictors from reusable components. Using public datasets for training and evaluation, we provide many examples of modular predictors and show how an effective predictor can be dynamically assembled during training from a network of modular components. Traditional intrusion detection systems (IDS), such as Snort [1] or Suricata [2], rely on rule-based configurations that are manually crafted and maintained by domain experts. The growing complexity and rapid evolution of IT systems make the maintenance of these rules increasingly challenging and time-consuming. As a response, research efforts into automated cyberdefence have started, based on the idea that attack patterns can be dynamically learned. The rules are no longer defined by humans, but automatically inferred from observing systems under attack. Over the last decade, various approaches have been proposed for automated cyberdefence, most of them based on statistical learning, e.g., [3], [4], [5], [6]. We follow this direction in the paper. We are specifically interested in predicting the stage of an ongoing attack in real time, based on current and earlier observations of an IT system.


Ultralight Polarity-Split Neuromorphic SNN for Event-Stream Super-Resolution

Xu, Chuanzhi, Zhou, Haoxian, Chen, Langyi, Chung, Yuk Ying, Qu, Qiang

arXiv.org Artificial Intelligence

Event cameras offer unparalleled advantages such as high temporal resolution, low latency, and high dynamic range. However, their limited spatial resolution poses challenges for fine-grained perception tasks. In this work, we propose an ultra-lightweight, stream-based event-to-event super-resolution method based on Spiking Neural Networks (SNNs), designed for real-time deployment on resource-constrained devices. To further reduce model size, we introduce a novel Dual-Forward Polarity-Split Event Encoding strategy that decouples positive and negative events into separate forward paths through a shared SNN. Furthermore, we propose a Learnable Spatio-temporal Polarity-aware Loss (LearnSTPLoss) that adaptively balances temporal, spatial, and polarity consistency using learnable uncertainty-based weights. Experimental results demonstrate that our method achieves competitive super-resolution performance on multiple datasets while significantly reducing model size and inference time. The lightweight design enables embedding the module into event cameras or using it as an efficient front-end preprocessing for downstream vision tasks.


EvRainDrop: HyperGraph-guided Completion for Effective Frame and Event Stream Aggregation

Wang, Futian, Zhang, Fan, Wang, Xiao, Wang, Mengqi, Huang, Dexing, Tang, Jin

arXiv.org Artificial Intelligence

Event cameras produce asynchronous event streams that are spatially sparse yet temporally dense. Mainstream event representation learning algorithms typically use event frames, voxels, or tensors as input. Although these approaches have achieved notable progress, they struggle to address the undersampling problem caused by spatial sparsity. In this paper, we propose a novel hypergraph-guided spatio-temporal event stream completion mechanism, which connects event tokens across different times and spatial locations via hypergraphs and leverages contextual information message passing to complete these sparse events. The proposed method can flexibly incorporate RGB tokens as nodes in the hypergraph within this completion framework, enabling multi-modal hypergraph-based information completion. Subsequently, we aggregate hypergraph node information across different time steps through self-attention, enabling effective learning and fusion of multi-modal features. Extensive experiments on both single- and multi-label event classification tasks fully validated the effectiveness of our proposed framework. The source code of this paper will be released on https://github.com/Event-AHU/EvRainDrop.


The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process

Hongyuan Mei, Jason M. Eisner

Neural Information Processing Systems

Many events occur in the world. Some event types are stochastically excited or inhibited--in the sense of having their probabilities elevated or decreased--by patterns in the sequence of previous events. Discovering such patterns can help us predict which type of event will happen next and when. We model streams of discrete events in continuous time, by constructing a neurally self-modulating multivariate point process in which the intensities of multiple event types evolve according to a novel continuous-time LSTM . This generative model allows past events to influence the future in complex and realistic ways, by conditioning future event intensities on the hidden state of a recurrent neural network that has consumed the stream of past events. Our model has desirable qualitative properties. It achieves competitive likelihood and predictive accuracy on real and synthetic datasets, including under missing-data conditions.